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ABSTRACT 

Organizations  engaged  in  international  operations  must  navigate  complex  intercultural  dynamics  for  successful 
performance,  necessitating  identification  of  individuals  who  are  likely  to  succeed  in  these  environments  and  training 
personnel  in  cross-cultural  competence  (3C).  To  do  so,  adequate  competency  models  of  3C  need  to  be  developed 
and  valid  3C  assessment  instruments  must  be  identified  or  generated.  The  present  chapter  reviews  issues  and 
challenges  in  3C  model  development  and  illustrates  these  problems  in  an  analysis  of  the  Defense  Language  Office’s 
Framework  for  Cross-Cultural  Competence.  The  comparative  advantages  of  competency  versus  causal  models  of  3C 
are  discussed  and  an  integration  of  competency  and  causal  models  is  suggested.  An  examination  of  34  instruments 
that  have  been  recommended  in  the  3C  civilian  and  military  literatures  for  assessing  cross-cultural  competencies  and 
their  antecedent  factors  showed  that  existing  3C  measures  suffer  from  poor  construct  validity  and  have  not  been 
empirically  linked  to  important  outcome  variables.  A  measurement  strategy  that  eschews  self-report  methods  and 
broadly  assesses  KSAOs  and  behavioral  competencies  is  advocated. 
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1  COMPETENCIES  FOR  CROSS-CULTURAL  READINESS 

Cross-cultural  competence  (3C),  often  defined  as  the  ability  to  quickly  understand  and  effectively  act  in  a  culture 
different  from  one’s  own  (e.g.,  Abbe,  Gulick,  &  Herman,  2007),  has  garnered  increasing  attention  within  the  U.S. 
Military.  Military  missions  have  become  more  complex,  blending  traditional  military  operations  with  nation 
building  goals  that  require  a  broader  set  of  competencies  over  a  greater  range  of  specialties  and  ranks,  than  the 
Military  was  previously  required  to  sustain  (Abbe,  2008).  The  U.S.  Department  of  Defense,  through  entities  such  as 
the  Defense  Equal  Opportunity  Management  Institute  (DEOMI),  the  Defense  Language  Office  (DLO),  and  the 
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Army  Research  Institute  (ARI),  has  devoted  substantial  resources  to  understanding  and  enhancing  3C.  Doing  so 
requires  resolving  three  issues:  (1)  What  is  3C,  or  more  specifically,  which  competencies  are  needed  by  which 
personnel,  and  at  what  level  of  performance?  (2)  How  can  these  competencies  be  assessed,  for  both  selection  or 
training  purposes?  and  (3)  How  can  these  competencies  be  trained?  The  present  chapter  addresses  the  first  two 
issues. 

Cross-cultural  competence  has  been  studied  actively  in  the  civilian  sector  since  the  1950s.  More  recently,  the 
Department  of  Defense  has  attempted  to  develop  a  conceptualization  of  3C  for  the  Military  through  several  official 
programs,  including  the  National  Security  Language  Initiative  (U.S.  Department  of  State,  2008),  the  Department  of 
Defense  Strategic  Plan  for  Language,  Regional,  and  Cultural  Capabilities  (2010),  and  the  Department  of  Defense 
Language  Transformation  Roadmap  (2005).  The  Roadmap  states  that  language,  culture,  and  regional  expertise  are 
not  only  important  “defense  core  competencies”  but  also  “critical  weapons  systems.” 

Extant  conceptualizations  of  3C  generally  include  four  components:  (1)  relatively  stable  characteristics  of  the 
individual,  such  as  personality  traits,  cognitive  capabilities,  social  competency,  and  cognitive  styles;  (2)  culture- 
general  and  region-specific  knowledge;  (3)  attitudinal  and  motivational  dispositions  such  as  ethnocentrism,  interest 
in  culture,  and  motivation  to  learn;  and  (4)  skills  such  as  communication,  language,  culturally  appropriate  behaviors, 
and  executive  functions  such  as  emotional  regulation  and  metacognition. 

Spitzberg  and  Changnon  (2009)  reviewed  the  many  models  of  3C  developed  for  business,  education,  the  Peace 
Corps  and  migration.  The  most  carefully  constructed  conceptualization  of  3C  for  the  U.S.  Military  at  this  time  is  the 
“Framework  for  Cross-Cultural  Competence.”  The  Framework  was  developed  in  several  stages  beginning  in  2008 
within  the  Defense  Regional  and  Cultural  Capabilities  Assessment  Working  Group  (RACCA  WG)  (McDonald, 
McGuire,  Johnston,  Selmeski,  &  Abbe,  2008).  RACCA  subject  matter  experts  (SMEs)  identified  a  set  of  40  general 
cross-cultural  learning  statements  consisting  of  knowledge,  skills,  and  personal  characteristics  (also  called  “core 
competencies”).  A  second,  overlapping  group  of  culture  experts  reduced  the  RACCA  competencies  to  a  smaller 
number  and  drew  a  distinction  between  antecedent  variables,  which  were  termed  “enablers,”  and  competencies 
(Johnston,  Paris,  McCoy,  Severe,  &  Hughes,  2010).  Johnston  et  al.  (2010)  was  revised  in  2011  and  the  Framework 
was  subsequently  revised  again  in  a  series  of  communications  within  a  group  of  DLO  culture  experts  in  March, 
2011. 

1.1  Competency  Models  of  3C 

The  Framework  is  one  of  the  most  recent  3C  models  and  reflects  two  mutually  supportive  trends:  (a)  an 
increased  emphasis  on  addressing  3C  as  an  applied  psychology  problem,  the  solution  to  which  has  important, 
material  implications  for  the  success  of  businesses  and  the  Military,  rather  than  a  theoretical  area  of  pursuit;  and  (b) 
the  inclusion  of  behavioral  capabilities  or  performance  domains  using  the  language,  if  not  the  full  development 
cycle,  of  competency  modeling  (Shippmann,  Ash,  Battista,  Carr,  Eyde  &  Hesketh,  et  al.  2000).  In  a  competency 
model  of  3C,  core  competencies  are  identified  in  a  hierarchical  categorization  system.  In  this  system,  general 
competencies  such  as  “cultural  perspective  taking”  are  used  to  form  categories  encompassing  more  specific 
competencies  that  are  defined  behaviorally,  for  example,  “understands  how  one’s  own  group  is  viewed  by  members 
of  another  group.”  Competency  potential  dimensions  (Bartram,  2005)  are  also  identified  in  such  models, 
incorporating  variables  that  have  been  called  antecedents  or  precursors  in  the  expatriate  and  overseas  study 
literatures.  These  variables  are  termed  enablers  in  the  Framework,  and  include  a  wide  range  of  characteristics  such 
as  personality  traits,  cognitive  abilities,  cognitive  style,  and  attitudes.  However,  unlike  antecedents  in  earlier  models, 
enablers  are  conceptualized  in  a  competency  modeling  (i.e.,  behavioral)  style. 

Competency  models  of  3C  such  as  the  DLO  Framework  may  be  subject  to  some  of  the  unresolved  problems  in 
competency  modeling  in  general.  Although  competency  modeling  is  popular  in  human  resource  management,  it 
suffers  from  a  great  deal  of  ambiguity  concerning  its  core  construct — competency — as  well  as  how  it  differs  from 
traditional  job  analysis  (Shippman  et  al.,  2000).  Morgeson,  Delaney-Klinger,  Mayfield,  Ferrara,  and  Campion 
(2004)  state,  “perhaps  one  of  the  most  vexing  issues  involves  actually  defining  a  competency”  (p.  676).  For 
example,  are  competencies  composed  of  KSAOs  (knowledge,  skill,  ability,  other),  or  are  KSAOs  antecedent  to 
competencies?  If  the  latter,  what  is  antecedent  to  KSAOs?  As  Van  de  Vijver  and  Leung  note,  “It  could  be  argued 
that  intercultural  competence  is  no  exception  to  the  rule  that  there  are  no  widely  shared  definitions  of  crucial 
concepts  in  psychology”  (2009,  p.  406).  Specifying  the  correct  number  of  competencies  and  their  organization  poses 
a  problem  for  competency  modeling  that  is  also  present  in  the  Framework.  For  any  given  military  occupational 


specialty  (MOS),  mission,  or  action,  which  competencies  are  important  and  how  many  can  be  practically  assessed? 

Competency  models  of  3C  share  several  limitations  that  persist  within  the  intercultural  adjustment  and 
performance  literature:  (1)  imprecision  in  defining  constructs,  often  in  the  absence  of  operationalization;  (2) 
conceptual  overlap  and  unsatisfactory  distinctions  among  key  model  components  such  as  antecedents,  KSAOs,  and 
performance  outcomes;  (3)  imprecision  in  specifying  the  causal  order  among  constructs;  and  (4)  imprecision  or  poor 
articulation  of  competencies  with  respect  to  the  U.S.  Military’s  practical  selection  and  training  needs  due  to 
insufficient  attention  to  MOS,  rank,  and  service  variables.  These  shortcomings  limit  the  predictive  and  explanatory 
ability  of  existing  3C  models,  and  consequently,  limit  the  predictive  ability  of  existing  3C  assessments,  making  them 
less  than  ideal  for  military  use.  It  is  critical  that  future  work  on  3C  focuses  explicitly  on  addressing  as  many  of  these 
concerns  as  possible. 

Gabrenya,  Moukarzel,  Pomerance,  Griffith  and  Deaton’s  (2011a)  in-depth  analysis  of  the  DLO  Framework 
competency  model  can  be  used  to  illustrate  some  of  these  issues.  The  current  version  of  the  DLO  Framework 
includes  five  competencies  and  seven  enablers.  Each  competency  is  defined  by  or  explained  by  sets  of  more  specific 
behaviors  and  skills.  These  definitional  items  are  themselves  considered  competencies,  introducing  a  hierarchical 
structure.  Some  definitional  items  are  in  turn  comprised  of  more  than  one  relatively  distinctive  component 
competency.  For  example,  the  competency  Culture  General  Concepts  and  Knowledge  includes  three  parts: 

•  Acquires... culture-general  concepts  and  knowledge 

•  Applies  culture  general  concepts  and  knowledge 

•  Comprehends... 

o  ...and  navigates... 

•  . .  .intercultural  dynamics 

Gabrenya  et  al.  (2011a)  refer  to  components  at  the  most  granular  level  as  elements.  The  imprecision  of  these 
elements  stems  from  the  Framework’s  use  of  colloquial  terms,  a  long-recognized  problem  throughout  social  and 
behavioral  science.  Most  readers  would  believe  they  know  what  “navigates”  means  in  this  context,  but  they  would 
probably  disagree  about  how  to  operationalize  or  assess  the  competency.  Similarly,  “intercultural  dynamics”  has 
strong  referents  in  common  life  experiences  but  is  not  useful  from  a  conceptual  or  measurement  perspective. 

The  overlapping  competency  issue  is  illustrated  by  considering  the  competency  Culture  Perspective  Taking, 
which  includes  four  distinguishable  components  and  some  subcomponents: 

•  Demonstrates  an  awareness  of  one’s  own  world  view  (i.e.  cultural  perceptions,  assumptions,  values,  and 
biases)  ... 

o  . .  .and  how  it  influences  our  behavior  and  that  of  others 

•  Understands  how  one’s  own  group  is  viewed  by  members  of  another  group 

•  Understands  ... 

o  ...and  applies... 

o  ...perspective-taking  skills  to  detect,  analyze,  and  consider  the  point  of  view  of  others  and 
recognizes  how  the  other  will  interpret  his/her  actions 

•  Takes  the  cultural  context  into  consideration  when  interpreting  situational  cues 

An  observer,  trainer,  or  evaluator  would  be  hard-pressed  to  distinguish  “understanding  and  applying  culture 
general  concepts  for  navigating  intercultural  dynamics”  in  the  Culture  General  Concepts  and  Knowledge 
competency  from  “applying  perspective-taking  skills  to  recognize  how  others  will  interpret  one’s  actions”  in  this 
competency. 

Competency  models  do  not  usually  attempt  to  specify  causal  relationships  among  competencies  or  between 
competencies  and  antecedent  variables.  Nonetheless,  such  causal  relationships  are  assumed  in  a  model  that  includes 
sets  of  competencies  and  enablers,  and  they  are  implicit  in  competency  hierarchies  that  specify  both  the  acquisition 
and  subsequent  application  of  knowledge  or  skills.  We  argue  in  a  later  section  that  causal  models  should  be 
explicitly  included  in  3C  conceptualizations. 

Most  serious  for  the  practical  application  of  cultural  expertise  in  the  Military  is  the  fourth  criticism,  insufficient 
attention  to  MOS,  rank,  and  service  variables.  This  problem  essentially  involves  establishing  the  appropriate  content 
of  the  model,  analogous  to  establishing  content  validity  in  test  development.  Whether  performed  from  a  classic  job 
analysis  perspective  (Brannick,  Levine  &  Morgenson,  2007)  or  from  a  competency  modeling  approach,  information 


must  be  obtained  to  attend  to  the  competencies,  selection  factors,  and  appropriate  training  of  3C  at  a  greater  level  of 
detail  than  found  in  existing  models.  Several  empirical  studies  have  been  utilized  to  inform  3C  requirements  in  the 
Military.  A  study  by  the  RAND  Corporation  (Hardison,  Sims,  Ali,  Villamizar,  Mundell,  &  Howe,  2009)  was 
conducted  to  help  conceptualize  training  program  content  to  improve  cross-cultural  performance  within  the  Air 
Force.  Unlike  most  such  studies,  the  RAND  study  incorporated  MOS  (AFSC-Air  Force  Specialty  Code)  and  rank, 
finding  considerable  variability  in  the  overall  importance  rating  of  3C  across  specialties.  Spencer  (2010)  reported  a 
qualitative  analysis  of  special  operations  forces  (SOF)  personnel  that  combined  features  of  competency  modeling 
and  job  analysis.  The  most  recent  investigation  of  3C  in  the  Military  at  the  time  of  this  writing  was  carried  out  by 
McCloskey,  Grandjean,  Behymer,  and  Ross  (2010)  using  respondents  who  had  returned  from  various  overseas 
postings.  Theoretical  approaches  to  establishing  the  content  of  3C  have  been  carried  out  by  Caligiuri,  Noe,  Nolan, 
Ryan,  and  Drasgow  (2011)  for  the  Military  and  by  Deardorff  (2006)  and  Hunter,  White  and  Godbey  (2006)  using  a 
Delphi  approach  based  on  the  combined  perspectives  of  multiple  subject  matter  experts  (SMEs)  concerning  civilian 
3C. 

1.2  An  Analysis  of  the  Adequacy  of  the  DLO  Framework 

Gabrenya  et  al.  (2011a)  used  these  and  other  sources  to  analyze  the  content  of  the  DLO  Framework.  They 
concluded  that  the  Framework  competencies  adequately  reflect  military  and  civilian  conceptions  despite  problems 
involving  overlapping  and  imprecisely  articulated  competencies.  The  Framework  was  found  to  have  19  enabler 
elements,  12  of  which  were  well  represented  in  other  conceptions  while  four  were  poorly  or  not  at  all  represented 
(e.g.,  “avoid  stress-induced  perspectives  that  oversimplify  culture,”  and  “acts  as  a  calming  influence”).  A  few 
competencies  were  not  found  in  the  Framework  despite  their  importance  in  the  civilian  competency  literature,  such 
as  ability  to  manage  family  obligations.  While  not  relevant  for  some  deployments,  family  and  spouse  adjustment  in 
the  foreign  context  is  the  strongest  predictor  of  success/failure  in  overseas  assignments  (Caligiuri,  Hyland,  Joshi,  & 
Bross,  1998).  The  issue  of  language  ability  has  been  discussed  in  various  documents.  Caligiuri  et  al.  (2011)  note  that 
teaching  and  maintaining  language  skills  is  not  cost  effective  for  the  Military,  but  training  3C  may  yield  a  better 
payoff.  The  RAND  Air  Force  study  (Hardison  et  al.,  2009)  found  both  low  valuation  of  language  skills  and  low 
language  capabilities:  4%  claimed  a  working  knowledge  of  the  language  of  the  place  to  which  they  had  been 
deployed,  and  10%  claimed  a  working  knowledge  of  any  foreign  language.  The  authors  suggest  that  low  proficiency 
may  have  led  to  low  valuation,  suggesting  that  self-reported  valuation  of  competencies  may  not  provide  a  good 
measure  of  their  actual  importance.  Language  acquisition  as  a  culture  competency  will  undoubtedly  remain  an 
ongoing  locus  of  debate  in  the  Military. 

The  McCloskey  et  al.  (2010)  study,  although  an  admirable  effort,  suffers  from  some  of  the  limitations  of  this 
kind  of  research  and  illustrates  some  of  the  dilemmas  in  using  active  duty  service  personnel  as  SMEs.  The  study 
included  an  overly  small  and  insufficiently  broad  sample,  which  from  the  start  limited  its  ability  to  analyze  its  SME 
data  in  terms  of  MOS  and  rank.  Like  all  qualitative  research,  it  had  to  navigate  the  problem  of  developing 
conceptual  categories  (in  this  case,  competencies  or  antecedents)  inductively  from  its  respondents’  information, 
while  using  the  large  existing  3C  literature  to  interpret  the  data.  Hence,  they  used  a  KSAO  set  generated  from 
previous  research  to  organize  the  respondents’  ideas  about  what  caused  or  constituted  effective  3C  when  they  were 
deployed.  In  any  given  time  frame,  most  deployed  personnel  will  experience  only  a  small  set  of  culture  regions.  For 
example,  at  the  present  time  most  deployed  personnel  are  in  the  Middle  East  or  at  bases  in  Korea,  Japan,  and 
Europe.  While  it  should  be  feasible  to  generate  a  competency  model  that  is  not  overly  biased  toward  a  particular 
culture  region  or  kind  of  deployment,  it  is  not  assured,  so  over  time  new  data  should  be  continually  obtained  and 
used  to  revisit  3C  models.  A  fundamental  problem  in  such  a  bottom-up  generation  of  KSAOs  and  competencies  is 
the  veridicality  of  the  relationship  between  the  KSAOs  reported  by  respondents  and  actual  performance.  Respondent 
data  such  as  that  generated  in  the  McCloskey  study  privileges  the  respondent’s  point  of  view,  based  as  it  is  on  the 
respondent’s  implicit  theories  about  the  relationships  between  antecedents  and  outcomes.  Research  in  social 
cognition  and  decision  making  has  shown  that  people  are  imperfectly  aware  of  the  causes  of  their  own  and  others’ 
behavior  and  success/failure  outcomes  (Fiske  &  Taylor,  2008).  Therefore,  research  is  needed  to  establish  the  causal 
links  in  integrated  models  once  valid  competency  and  performance  measures  are  developed. 

The  problem  of  identifying  the  correct  source  information  on  which  to  build  3C  models  cannot  be  separated  from 
the  bottom-line  question  of  which  antecedents  do  in  fact  “enable”  which  competencies,  and  which  competencies  in 
turn  actually  affect  performance.  The  situationally  labile  nature  of  performance,  especially  in  complex  missions 


involving  many  units  and  on-the-ground  factors,  poses  considerable  difficulties  to  researchers  who  prefer  to  settle 
the  antecedent -competency-performance  relationships  empirically.  Gabrenya  et  al.  (2011a)  called  on  the  large 
civilian  literature  to  examine  the  relationships  between  DLO  Framework  enablers  and  competencies  on  the  one 
hand,  and  performance  on  the  other.  Their  strategy  was  to  interpret  the  competency  and  enabler  elements  in  terms  of 
constructs  or  variables  that  have  been  the  subject  of  research  on  intercultural  effectiveness  (i.e.,  map  elements  to 
constructs),  find  good  measures  of  these  constructs,  and  then  review  the  research  on  the  criterion  validity  of  the 
measures  that  used  performance  or  adjustment  criterion  variables.  The  strategy  looks  like  this: 

Competency  or  Enabler  Element  ->  Representative  construct  ->  Valid  measure  ->  Performance/Adjustment  Criterion 

For  example,  the  Framework  enabler  element  receptive  to  new  ways  of  doing  things  maps  to  constructs  such  as 
flexibility  and  openness  which  in  turn  can  be  assessed  by  several  instruments.  Due  to  the  manner  in  which  elements 
were  often  expressed  behaviorally,  vaguely,  or  in  mission-specific  terms,  the  mapping  phase  was  imprecise. 

We  performed  a  comprehensive  search  of  the  sojourner  adjustment/performance  literature  to  identify  measures 
that  could  be  used  in  this  evaluation.  Our  search  capitalized  on  other  attempts  to  create  comprehensive  lists  of 
instruments,  for  example  Fantini  (2009),  Thomson  and  Ross  (2008),  the  website  of  the  Institute  for  Intercultural 
Training  (www.intercultural.org),  and  the  websites  of  several  consulting  companies.  In  addition  to  instruments  that 
were  developed  specifically  for  cultural  research  purposes,  we  also  looked  at  studies  that  used  familiar  personality 
instruments  in  the  large  sojourner  adjustment  literature,  such  as  the  NEO,  coping  style  scales,  and  measures  of  social 
interaction  individual  differences  (e.g.,  the  Self-Monitoring  Scale).  We  return  to  our  evaluation  of  the  state  of 
measurement  in  this  field  in  another  section. 

The  constructs  represented  by  five  of  the  12  competency  elements  and  five  of  the  19  enabler  elements  could  not 
be  measured  because  either  (1)  no  suitable  instrument  has  been  developed,  or  (2)  the  available  instruments  have 
never  been  used  in  concurrent  or  predictive  validity  studies  involving  intercultural  adjustment  of  performance 
variables.  Of  the  seven  measurable  competency  elements,  six  were  supported  and  one  received  mixed  evidence.  Of 
the  12  measurable  enabler  elements,  four  were  well  supported,  six  received  mixed  support,  and  two  received  no 
support.  It  must  be  noted  that  this  is  a  conservative  analysis  in  that  (1)  better  instrumentation  would  afford 
evaluation  of  more  elements;  (2)  additional  research  might  be  able  to  produce  more  conclusive  findings  when 
evidence  was  mixed  by  sorting  out  which  instruments  best  assessed  the  construct;  and  (3)  most  importantly, 
elements  that  did  not  map  well  against  constructs  (in  one-to-many  and  many-to-one  relationships)  need  to  be 
additionally  decomposed  if  model  adequacy  is  to  be  established  empirically. 

The  findings  of  this  evaluation  of  an  important  military  3C  model  are  not  encouraging  in  several  respects.  First, 
we  could  not  find  valid  measures  of  a  large  number  of  elements  despite  a  wide-ranging  search,  and  many  of  the 
instruments  that  we  did  find  did  not  demonstrate  adequate  psychometric  properties.  Second,  the  competency  model 
nature  of  the  Framework  rendered  it  poorly  amendable  to  empirical  validation.  Third,  our  mapping  project  illustrated 
an  intrinsic  weakness  of  competency  models,  an  issue  to  which  we  now  turn. 

2  COMPETENCY  MODELS,  CAUSAL  MODELS,  AND  INTEGRATED  MODELS 

The  DLO  Framework  may  be  termed  a  compositional  model  in  the  sense  used  by  Spitzberg  and  Changnon 
(2009)  in  their  attempt  to  categorize  the  disparate  models  of  sojourner  adjustment  and  expatriate  performance.  Such 
models  are  primarily  lists  of  KSAOs  that  comprise  3C,  usually  organized  in  logical  sets  in  a  way  that  implies  a 
causal  sequence.  The  Framework  uses  descriptions  of  job-related  behaviors,  in  the  manner  of  a  typical  competency 
model,  rather  than  constructs  or  variable  names,  to  describe  the  set  of  desired  qualities.  In  contrast  to  list-like 
compositional  models,  causal  path  models  represent  a  linear  causal  system  that  may  or  may  not  involve  feedback 
paths;  such  models  can  usually  be  tested  using  multivariate  methods  (Spitzberg  &  Changnon,  2009).  Causal  path 
models  are  familiar  to  social  scientists  who  primarily  create  models  to  represent  individual  and  social  processes  and 
to  generate  testable  hypotheses;  hence,  they  are  probably  the  most  common  type  of  model  found  in  this  field.  Good 
models  approximate  miniature  theories;  they  are  tentative  and  falsifiable  (Graziano  &  Raulin,  2004). 

Competency  models  have  advantages  and  disadvantages  compared  to  causal  models.  A  competency  model 
provides  generalizable  guidance  for  training,  selection,  and  assessment,  and  is  therefore  directed  to  solving  an 
applied  psychology  problem,  such  as  enhancing  3C  capabilities  in  the  Military.  Causal  models,  however,  provide 
conceptual,  theoretical,  and  research  advantages  that  can  guide  selection  and  training  by  showing  where  in  the 


antecedent-to-competency  relationship  they  should  be  used  to  greatest  effect.  A  competency  model  without  causal 
validation  may  be  difficult  to  generalize  to  different  contexts  because  it  lacks  a  nomological  network  of  meditational 
and  moderator  constructs  that  can  take  into  account  contextual  or  situational  variation. 


2.1  An  Integrated  Model  Approach 

These  two  approaches  may  be  reconciled,  however,  if  competencies  and  antecedents  were  integrated  in  models 
that  show  causality,  mediation,  and/or  moderation.  Each  higher  level  or  general  competency  would  be  embedded  in 
a  model,  and  the  competencies  would  be  related  to  each  other  in  larger  models.  Valid  measures  of  the  competencies 
would  need  to  be  identified  or  developed  that  assess  a  wide  range  of  competencies  and  antecedents  beyond  cognitive 
measures.  Figure  1  illustrates  how  such  an  integrated  model  could  be  developed  using  competencies  and  enablers  in 
the  DLO  Framework. 

One  final  critical  advantage  of  the  integrated  model  approach  is  that  it  provides  multiple  assessment  points  for 
evaluating  the  3C  competency  of  individuals.  An  integrated  model  reveals  antecedents  that,  because  they  are  often 
more  easily  measured  than  complex  competencies,  can  be  assessed,  along  with  the  competency  itself,  to  provide  a 
profile  of  the  knowledge,  skills,  abilities,  and  other  characteristics  of  the  individual  that  “triangulates”  on  the 
person’s  overall  level  of  cross-cultural  proficiency.  Although  the  ideal  tool  for  assessment  of  behavioral 
competencies  is  the  classic  assessment  center,  this  method  is  costly  and  time  consuming.  By  including  antecedent 
measurement  in  the  mix  of  3C  assessments,  the  need  to  use  full  assessment  center  methods  to  obtain  behavioral 
observations  of  competencies  is  reduced.  Instead,  simpler  and  more  efficient  “mini  assessment  centers”  can  be 
developed  to  add  behavioral  assessment  to  the  available  information.  These  simplified  behavioral  measures  would 
not  require  assessors  and  would  be  designed  for  administration  via  the  Internet  in  batteries  that  include  self-report 
measures  of  antecedent  variables. 


Figure  1 .  Example  of  integrated  model  surrounding  perspective  taking  as  a  competency 


Therefore,  we  suggest  that  an  integrated  causal-competency  approach  should  be  utilized  in  developing  a  new  3C 
model.  The  integrated  3C  model  should  be  built  as  a  set  of  causal  models  drawn  from  existing  theory  and  research, 
each  designed  to  understand  a  single  3C  competency.  Each  competency  will  be  conceptualized  in  a  nomological 
network  of  antecedents,  competencies,  and  performance  outcomes  drawn  from  prior  research  that  includes  the  most 
important  moderators,  articulated  as  needed  by  MOS/rank/service.  Cross-sectional  and  longitudinal  correlational 
studies,  as  well  as  experimental  training  studies,  can  be  conducted  to  test  such  an  integrated  model.  In  this  way,  the 
applied  usefulness  of  a  competency  model  can  be  retained,  while  a  research  capability  is  added  that  can  test  the 
adequacy  of  the  proposed  competencies.  Through  a  process  of  refinement  the  result  should  be  a  robust  competency 
framework  that  can  guide  selection  and  training  efforts. 

The  integrated  model  we  advocate  is  measurement  intensive:  we  want  to  measure  everything  in  the  model,  if 
possible,  in  addition  to  moderators  that  prove  important.  A  serious  problem  we  encountered  in  our  attempt  to 
establish  the  criterion  validity  of  the  elements  of  the  DEO  Framework  was  the  paucity  of  good  measures  for 
elements  that  one  would  expect  should  be  measureable,  and  the  complete  absence  of  measures  of  many  behavioral 
competencies.  In  the  next  section,  we  discuss  the  state  of  measurement  and  suggest  some  new  avenues  for 


assessment. 


3  ASSESSING  CROSS-CULTURAL  COMPETENCE 

3 . 1  Validity  of  Existing  Instruments 

Our  evaluation  of  the  criterion  validity  of  the  DLO  Framework  necessitated  identifying  instruments  to  assess  the 
constructs  to  which  the  Framework  elements  were  mapped  and  then  finding  studies  that  used  these  instruments 
alongside  acceptable  criterion  variables,  i.e.,  performance  and  adjustment  measures.  In  this  second  step,  we  were 
essentially  evaluating  the  quality  of  much  of  the  existing  3C  instrument  domain.  Because  of  the  manner  in  which  we 
tracked  down  instruments  (from  others’  lists,  primarily),  we  were  secondarily  evaluating  the  quality  of  these 
ubiquitous  lists.  We  discovered  that  a  variety  of  performance  measures  are  used  in  this  literature,  including  job 
performance  (manager  ratings,  peer  ratings,  self-ratings)  and  several  informal  ratings  of  overseas  “success”  or 
“effectiveness”  in  non-job  situations.  Expatriate  “performance”  is  also  assessed  indirectly  through  self-reports  of 
intent  to  remain  on  the  job,  job  attitudes,  and  occupational  citizenship  behaviors  (Mol,  Bom,  Willemsen,  &  van  der 
Molen,  2005;  Thomas  &  Lazarova,  2006).  Proprietary  instruments  were  difficult  to  evaluate;  validation  studies  have 
been  published  in  peer  reviewed  journals  for  only  a  few  such  instruments  and  satisfactory  validation  reports  are 
rarely  published  on  consulting  companies’  websites.  Altogether,  we  identified  34  instruments. 

Gabrenya  et  al.  (2011a)  present  the  results  of  this  analysis  in  detail.  The  criterion  validity  with  respect  to 
intercultural  performance  or  adjustment  of  five  of  the  instruments  was  good,  for  two  it  was  moderate  (subscales 
differed  in  quality),  for  25  it  was  poor  or  no  validation  information  could  be  found,  and  two  turned  out  not  to  be 
quantitative  instruments  “as  advertised”  in  instrument  lists.  In  several  cases,  the  instruments  appeared  to  have  good 
construct  validity  but  unproven  or  untried  criterion  validity,  holding  out  hope  that  future  research  would  lead  to 
discovering  some  additional  usable  instruments.  The  best  overall  instrument  in  terms  of  face,  construct,  and  criterion 
validation  appears  to  be  the  Multicultural  Personality  Questionnaire  (MPQ;  Van  Oudenhoven  &  Van  der  Zee,  2002). 

3.2  Need  for  New  Instrumentation 

The  list  of  instruments  evaluated  in  the  previous  section  attests  to  the  tremendous  effort  that  researchers  have 
devoted  to  developing  instruments  assessing  3C,  but  our  analysis  demonstrates  that  the  results  of  their  efforts  are 
discouraging,  so  far.  The  following  overlapping  problems  can  be  identified  in  the  existing  instrument 
armamentarium:  (1)  nearly  all  use  self-report  methods  that  appear  unsuitable  for  assessing  most  competencies;  (2) 
only  declarative,  cognitively  accessible,  and  self-referent  information  is  usually  obtained;  (3)  the  potential  for  faking 
ranges  from  subtle  to  severe;  (4)  affective  states  or  processes  are  poorly  assessed;  (5)  behavior  is  rarely  measured; 
(6)  the  instruments  map  poorly  to  DLO  Framework  competencies;  and  (7)  few  were  found  to  be  adequately 
validated  using  performance  criteria. 

These  issues  significantly  limit  the  validity  and  construct  coverage  of  existing  measures.  For  instance,  relying  on 
assessments  that  target  explicit,  declarative  knowledge  neglects  important  characteristics  that  are  more  implicit  or 
procedural  in  nature  (e.g.,  affective  and  skill-based  variables).  The  self-report  approach  used  in  existing  measures 
may  not  provide  sufficiently  valid  assessments  of  key  constructs.  Faking,  for  example,  is  one  major  issue  with  self- 
report  measures  in  many  situations  where  respondents  are  motivated  to  make  a  good  impression.  Numerous  studies 
have  indicated  that  individuals  are  able  to  substantially  alter  their  scores  these  measures  (e.g.,  Birkeland,  Manson, 
Kisamore,  Brannick,  &  Smith,  2006;  Viswesvaran  &  Ones,  1999)  and  that  a  sizable  proportion  do  fake  in  motivating 
situations  (e.g.,  Griffith  &  Converse,  2011).  Further  evidence  has  indicated  this  behavior  has  negative  implications 
for  construct  and  criterion-related  validity  (e.g..  Converse,  Peterson,  &  Griffith,  2009).  Another  concern  with  these 
types  of  self-reports  is  that  respondents  may  not  be  able  to  provide  accurate  self-assessments  of  knowledge-,  skill-, 
ability-,  or  competency-related  constructs  relevant  to  cross-cultural  competence.  Thus,  even  if  there  is  little 
motivation  to  intentionally  distort  responses,  self-report  measures  of  these  variables  may  not  provide  valid 
assessments.  Indeed,  a  number  of  studies  have  indicated  self-reports  of  knowledge,  skills,  abilities,  or  competencies 
have  limited  validity  (e.g..  Carter  &  Dunning,  2008). 

Consistent  with  these  ideas,  self-reports  of  cross-cultural  skills  and  abilities  have  been  criticized  on 
methodological  grounds  and  may  have  questionable  validity  (see  Brackett,  Rivers,  Shiffman,  Lerner,  &  Salovey, 


2006;  Gabrenya,  van  Driel,  Culhane,  Turner,  Pathak,  &  Peterson,  2011b;  Thomas  et  al.,  2008).  For  example, 
Gabrenya,  et  al.  (2011b)  examined  the  validity  of  the  Cultural  Intelligence  Scale  (CQS;  Ang  &  Van  Dyne,  2008). 
The  CQS  uses  self-reports  of  competencies,  attitudes,  and  behaviors  to  infer  cultural  intelligence,  the  ability  to 
generate  appropriate  behavior  in  new  cultural  settings  (Earley  &  Ang,  2003).  They  found  that  the  CQS  fails  to 
mediate  between  antecedent  variables  assessed  using  objective  and  behavioral  measures  and  criterion  variables  such 
as  situational  judgment  tests  and  adjustment.  The  CQS  appears  to  unintentionally  assess  constructs  such  as  self- 
efficacy  rather  than  cultural  intelligence  or  competence. 

The  considerations  listed  above  indicate  there  is  a  critical  need  to  develop  alternative  measurement  approaches  to 
assess  3C.  Comprehensive  measurement  of  3C  involves  three  sets  of  variables:  (1)  the  antecedent  variables  that 
comprise  each  competency’s  causal  model,  such  as  the  DLO  Framework  enablers;  (2)  the  individuals’ 
competencies;  and  (3)  the  performance  outcomes  through  which  criterion-related  validation  of  the  competency 
models  is  accomplished. 

We  propose  that  the  development  of  new  measures  should  be  based  on  two  general  principles  for  improving 
overall  measurement:  (1)  broadening  the  measurement  spectrum  and  (2)  moving  toward  more  dynamic 
measurement.  As  noted  earlier,  a  major  limitation  of  existing  3C  measures  is  that  they  tend  to  have  a  strong 
cognitive  focus,  with  most  involving  self-reports  of  declarative,  cognitively  accessible,  self-referent  information. 
Cultural  competence  involves  not  only  cognitive  factors,  but  also  affective  (e.g.,  emotion  regulation)  and  behavioral 
(e.g.,  interpersonal  skills)  components.  Thus,  future  measure  development  should  expand  the  focus  of  assessment  to 
include  cognitive,  affective,  and  behavioral  approaches  in  order  to  more  closely  and  accurately  match  the  full  set  of 
competencies  and  antecedents  in  a  well  developed  competency  model. 

Another  key  limitation  of  existing  3C  measures  is  that  they  are  also  largely  static  in  nature,  where  individuals 
respond  to  sets  of  items  under  typical  testing  conditions.  However,  cross-cultural  experiences  and  interactions  are 
generally  much  more  complex  and  dynamic,  requiring  more  active  behavior  (e.g.,  decision  making  and  self¬ 
regulation)  and  involving  elements  such  as  distraction,  stress,  ambiguity,  and  emotion.  Thus,  instrument 
development  must  incorporate  more  dynamic  assessment  approaches.  In  this  context,  we  use  the  term  dynamic  in  a 
general  sense  to  refer  to  alternatives  to  traditional  self-reports  that  involve  richer  stimuli,  changing  task 
characteristics,  and/or  more  active  involvement  from  the  individual,  i.e.,  behavioral  responding. 

4  CONCLUSION 

In  summary,  many  models  of  3C  have  been  proposed  across  decades  of  military  and  civilian  research  and 
theorizing.  Most  recently,  competency  frameworks  of  3C  that  are  aimed  at  informing  selection  and  training  of 
personnel  have  taken  the  forefront.  These  models  have  the  potential  to  solidify  an  amorphous  research  area  and  aid 
multi-national  organizations  as  they  encounter  new  cultures.  However,  the  existing  frameworks  suffer  from 
conceptual  ambiguity,  a  lack  of  causal  linkages  between  enablers  and  competencies,  and  inadequate  assessment 
measures.  To  advance  the  study  of  3C  and  provide  organizations  with  the  tools  to  make  important  personnel 
decisions,  integrated  models  of  3C  that  incorporate  competencies  with  traditional  causal  modeling  are  necessary.  In 
addition,  psychometrically  sound  measures  that  rely  less  on  self-report  methods  must  be  developed  if  these  models 
are  to  be  tested.  While  the  construct  of  3C  has  intuitive  appeal  to  address  many  of  the  challenges  facing  21st  century 
organizations,  operational  measures  and  models  of  3C  have  far  to  go  before  they  meet  that  promise. 
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